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TRAC  FY15  Research  Planning  and  Elicitation 

Chapter  1 
Introduction 


Overview 

This  document  provides  the  plan,  results  and  recommendations  for  improvement  for  the  fiscal 
year  2015  (FY15)  research  priorities  elicitation,  carried  out  by  TRAC — Monterey  (TRAC — 
MTRY)  in  support  of  the  TRAC  methods  and  research  office  (TRAC-MRO)  and  the  TRAC 
research  council. 


Background 

At  the  end  of  FY13,  TRAC-MTRY  conducted  a  research  priorities  elicitation  from  across 
TRAC  in  order  to  support  the  research  council’s  FY14  priorities  briefing  to  the  TRAC  board 
of  directors  (BOD).  This  plan  for  the  FY15  research  elicitation  captures  the  lessons  learned 
from  the  FY14  elicitation.  See  Marks  and  NesbittQfor  more  details  see. 


Research  Council 

As  topics  are  identified  and  prioritized,  the  Research  Council  identifies  a  lead  agency  for 
each  project  and  the  availability  and  level  of  resourcing.  The  Research  Council  serves  as 
a  review  agency,  in  conjunction  with  the  TRAC  stakeholder  that  originally  generated  the 
requirement,  for  products. 


Problem  Statement 

NLT  SEP  2014,  TRAC  research  council  elicits  and  prioritizes  research  requirements  from 
OneTRAC  in  order  to  provide  the  board  of  directors  (BOD)  with  a  recommended  set  of 
FY15  research  tasks. 


Christopher  Marks  and  Peter  Nesbitt.  TRAC  FY14  Research  Requirements  Elicitation.  Tech.  rep. 
TRAC-M-TM-13-059.  700  Dyer  Road,  Monterey,  CA  93943-0692:  Training  and  Doctrine  Command  Analy¬ 
sis  Center — Monterey,  2013.  URL:  https://hq.tradoc.army.mil/sites/trac/Projects/976/SitePages/ 
Home . aspx. 
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Name 

Title 

TRAC  Center 

Mr.  Paul  Works 

TRAC  Research  Director 

TRAC  HQ 

Mr.  Cody  Beck 

Scenario  Enterprise  Lead 

TRAC-FLVN 

Mr.  Eric  Johnson 

M&S  Enterprise  Lead 

TRAC-FLVN 

Mr.  Chad  Mullis 

Data  Enterprise  Lead 

TRAC-WSMR 

Dr.  Sylvia  Achionne-Noel 

Senior  Analyst 

TRAC-WSMR 

Mr.  Moe  Hayes 

Modeling  and  Analysis 

TRAC-LEE 

Mr.  Leroy  Jackson 

TRAC  Technical  Director  &  KM  En¬ 
terprise  Lead 

TRAC-MTRY 

Dr.  Chris  Morey 

TRAC  Study  Directorate 

TRAC 

LTC  Smith 

Director,  TRAC-MTRY 

TRAC-MTRY 

Table  1.  TRAC  Research  Council. 

Note:  the  titles  reflect  the  construct  of  the  research  council  when  this  project  started.  It  is 

not  the  current  TRAC  research  council. 


Issues  for  Analysis 

Issue  1:  What  are  TRAC’s  research  requirements  for  FY15? 

EEA  1.1:  What  are  the  research  objectives? 

Issue  2:  What  are  the  research  topics,  tools  and  methodologies  for  TRAC  in  FY15? 

EEA  2.1:  Which  research  requirements,  if  addressed,  have  the  potential  to  provide 
the  most  benefit  to  TRAC  studies? 

Constraints,  Limitations,  and  Assumptions 

•  Constraints 

-  UFR’s  due  September  2014. 

—  BOD  IPR  September  2014. 

•  Limitations 

•  Assumptions 


Constraints  limit  the  project  team’s  options  to  conduct  the  research.  Limitations  are  a 
project  team’s  inabilities  to  investigate  issues  within  the  sponsor’s  bounds.  Assumptions  are 
research-specific  statements  that  are  taken  as  true  in  the  absence  of  facts. 
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Technical  Approach 

The  technical  approach  for  this  elicitation  cycle  is  presented  in  Figure  [l}  The  numbers  in 
the  figure  represent  the  components  while  the  arrows  display  the  flow  of  information  from 
the  effort.  The  timeline,  which  is  called  out  in  the  tan  box,  is  articulated  in  the  next  section. 


Research  Elicitation  Plan 


TRAC  center 
interviews 


Research 

Council 


Timeline 

16  April-10  May  2014 
Mission  Analysis  and 
complete  elicitation  plan. 

11  May  -  30  June  2014 
Execute  elicitation. 

July  2014 

Analyze  elicitation  results. 

August  2014  -  Research 
council  explores  results. 

September  2014  - 
Recommended  results 
presented  to  the  BOD  for 
approval. 


Figure  1.  FY15  Research  Elicitation  Plan 


The  components  of  the  elicitation  process  are: 


1.  TRAC  leader  interviews  conducted  in  person  at  each  center. 

2.  Volunteer  surveys  open  to  all  TRAC  via  SharePoint. 

3.  Research  council  discussion  of  results  and  formulation  of  recommended  FY15  TRAC 
research  objectives 

4.  BOD  shown  results  of  elicitation  and  approval  of  research  objectives 

5.  TRAC  FY15  research  plan  published  with  research  objectives 


Steps  1  and  2  are  covered  in  detail  in  Chapter  [2  while  steps  3  and  4  along  with  the  approved 
research  objectives  are  in  Chapter  |3j  Chapter  4]  concludes  this  report  with  lessons  learned 
and  recommendations  for  the  next  iteration  of  the  elicitation  process. 
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Timeline 


16  April  -  10  May  2014 
11  May  -  30  Jun  2014 
July  2014 
August  2014 

September  2015 


Mission  analysis  and  complete  elicitation  plan 
Execute  elicitation 
Analyze  elicitation  results 

Research  council  explores  results  and  creates  recom¬ 
mended  research  objectives  for  FY15 
Recommended  research  objectives  and  elicitation  results 
presented  to  the  BOD  for  approval 
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Chapter  2 

Interview  and  Survey  Elicitation 


Leader  Interviews 

The  Process 

Leader  interviews  are  an  important  part  of  the  elicitation  process.  The  leadership  of  TRAC 
drive  the  strategy  and  are  invloved  in  implementing  the  vision  of  not  only  TRAC  leadership 
but  that  of  TRADOC  and  the  Army.  The  are  in  key  positions  to  project  the  future  capa¬ 
bilities  and  research  topics  in  which  we  should  be  exploring  to  maintain  TRAC’s  analytic 
relevance. 

In  order  to  interview  the  leadership,  we  first  defined  the  bounds  on  the  leaders  we  targeted  for 
interviews.  Since  each  TRAC  center  is  uniquely  structured,  we  decided  to  allow  directorate 
chiefs  at  FLVN  and  WSMR  as  well  as  division  chiefs  at  LEE  to  identify  the  leaders  they 
wanted  present  at  the  interviews.  We  discuss  the  directorates  and  divisions  interviewed  in 
the  subsequent  subsection.  We  used  the  questionnaire  presented  in  Appendix  [D]  to  frame 
our  discussions  as  prescriptive  guides. 

Most  interviews  were  completed  in  offices  or  local  conference  rooms  located  at  the  perspec¬ 
tive  centers.  They  were  informal  settings  in  which  the  center  host  selected  a  team  to  be 
interviewed  from  those  personnel  in  which  they  supervised.  The  SES  directors,  center  Se¬ 
nior  Military  Analyst  (SMA),  and  LEE  were  the  exceptions  in  which  they  were  interview 
individually.  MAJ  Mike  Teter,  the  research  coordinator  for  MTRY,  and  LTC  Chris  Smith, 
MTRY  Director,  were  present  at  all  interviews  to  ensure  there  were  two  sets  of  notes  in 
which  elicited  feedback  could  be  captured. 

Leader  Interviews  Schedule 

TRAC-HQ  personnel  or  divisions  interviewed: 


•  Director  -  Ms.  Blechinger 

•  Methods  and  Research  Office  -  Mr.  Works 


The  TRAC-HQ  leader  interviews  were  completed  the  12th  and  13th  of  May,  2014.  The 
results  are  covered  in  Chapter  [3]  and  the  specific  notes  taken  are  in  appendix  [Cj 
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TRAC-FLVN  Leader  Interviews 


TRAC-FLVN  personnel  or  divisions  interviewed: 

•  Director  -  Mr.  Pippen 

•  outgoing  and  incoming  Deputy  Director  and  SMA  -  COL  Arnhart  and  COL  Roller, 
respectively 

•  Analysis  Directorate  -  Dr.  Morey 

•  Studies  Directorate  -  Mr.  Decker 

•  Wargaming  and  Simulation  Directorate  -  Mr.  Johnson 

•  Operations  Directorate  -  Ms.  Fratzel 

•  Scenarios  and  Data  Directorate  -  Mr.  Beck 

The  TRAC-FLVN  leader  interviews  were  completed  the  12th  and  13th  of  May,  2014.  The 
results  are  covered  in  Chapter  [3]  and  the  specific  notes  taken  are  in  appendix  |C| 

TRAC-WSMR  Leader  Interviews 

TRAC-WSMR  personnel  or  divisions  interviewed: 

•  Director  -  Dr.  Lambert 

•  Deputy  Director  and  SMA  -  COL  Larimer 

•  Studies  and  Analysis  1  Directorate  -  Mr.  Eaton 

•  Studies  and  Analysis  If  Directorate  -  Mr.  Solis 

•  Studies  and  Analysis  Ill  Directorate  (to  include  the  Forward  Analysis  Division  -  Fort 
Bliss)  -  Mr.  Gard 

•  Modeling  and  Simulation  Directorate  -  Mr.  Mullis 

•  Study  Support  Directorate  -  Mr.  Huskey 

The  TRAC-WSMR  leader  interviews  were  completed  the  9th  through  the  lltli  of  June,  2014. 
The  results  are  covered  in  Chapter  [3]  and  the  specific  notes  taken  are  in  appendix  |C| 

TRAC-LEE  Leader  Interviews 

TRAC-LEE  personnel  or  divisions  interviewed: 

•  Director  -  Mr.  Byrd 
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•  Modeling  and  Analysis  Division  -  Mr.  Hayes 

•  Studies  Division  -  Mr.  Hopson 

The  TRAC-LEE  leader  interviews  were  completed  the  16th  of  June,  2014.  The  results  are 
covered  in  Chapter  [3]  and  the  specific  notes  taken  are  in  appendix  |Cj 


OneTRAC  Elicitation 


For  the  OneTRAC  elicitation,  we  constructed  a  survey  which  was  distributed  over  SharePoint 
and  was  open  for  the  entire  TRAC  workforce  from  the  12th  of  May  through  thel5th  of  July 


2014. 


Survey  Construction 

For  the  construction  of  the  survey,  we  used  free  text  fields  for  the  elicitation  in  an  effort 
to  ensure  no  intellectual  anchoring  among  respondents.  Once  someone  decided  to  take  the 
survey  and  clicked  through  the  SharePoint  link,  the  survey  was  a  simple  process  which  opened 
with  a  warning  screen  for  the  respondent  to  submit  the  survey  at  the  end.  (Displayed  in 
Figure  [Tj) 


Figure  1.  First  Page  of  Survey 


The  second  page  of  the  survey,  displayed  in  Figure  |2j  is  the  research  elicitation  in  which  the 
respondent  was  asked  to  give  a  short  sentence  about  the  research  requirement  followed  by 
a  lengthy  description.  The  raw  results  from  the  SharePoint  survey  are  in  Appendix  [Bj  The 
respondent  is  also  asked  to  identify  which  TRAC  center  would  be  a  stakeholder  and  what 
the  priority  of  research  should  be  in  regards  to  OneTRAC. 

Following  completion  of  the  the  second  page,  the  respondent  is  asked  if  they  have  another 
research  requirement.  (See  Figure  |3j) .  If  they  answer  yes  they  will  be  give  a  new  page  similar 
to  that  presented  in  Figure  [2]  in  which  they  have  the  opportunity  to  identify  more  research 
requirements.  Each  respondent  can  submit  up  to  four  research  requirements  per  survey.  The 
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Figure  2.  Second  Page  of  Survey 

respondents  were  also  not  limited  to  only  one  survey  but  could  submit  multiple  surveys 
they  had  more  than  four  research  requirements  to  submit. 


FY15  TRAC  Research  Elicitation  - 


Next  save  and  uose 


“  indicates  a  required  field 


Do  you  have  an  additional  research  requirement?  * 

|yes/vJ 


Qeated  at  2/17/2015  9:45  AM  by  **'  Next  I  Save  and  Close  I  Cancel 

Last  modified  at  2/17/2015  9:48  AM  by  ***  _ _ _ I _ _ _ I _ 


Figure  3.  Third  Page  of  Survey 


If  the  respondent  did  not  have  another  research  requirement  the  would  be  directed  to  the  last 
page  of  the  survey  (Figure  |4j)  in  which  they  were  asked  their  pay  grade,  which  TRAC-cneter 
they  were  assigned  and  if  they  had  previously  submitted  a  different  survey. 


Figure  4.  Last  Page  of  Survey 

Once  the  surveys  were  complete,  we  clustered  results  by  general  topics  and  describe  in  detail 
in  Chapter  [3] 


SharePoint  Survey  Demographics 

The  demographics  of  the  participants  who  chose  to  fill  out  the  survey  were  tracked  by  two 
categories,  pay  grade  and  TRAC-center.  The  pay  grades  are  displayed  in  Figure  [5]  and  the 
center  participation  is  displayed  in  Figure  |6j 
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|  Rank/Grade 

Number  of  surveys  \ 

06/GS15 
and  above 

1 

05/GS13-14  5 

04/GS12  4 

03/GS11  2 

and  Below 


Figure  5.  Pay  Grade  of  Survey  Respondents 


|  Center 

Number  of  surveys  j 

HQ 

2 

FLVN 

LEE 

MTRY 

WSMR 


3 

2 

2 

3 


Figure  6.  TRAC-Center  of  Survey  Respondents 
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Chapter  3 

Results  from  Research  Elicitation 


We  first  clustered  the  results  into  categorical  types.  We  extrapolated  research  objectives  from 
the  types  of  topics  and  the  stakeholder  of  the  topic.  We  did  this  by  allowing  one  “vote”  per 
live  interview  session  while  counting  the  SharePoint  survey  results  as  one  collective  interview 
session  for  voting  purposes.  In  this  Chapter,  we  cover  the  recommended  research  objectives 
and  supporting  count  of  topics  within  the  research  objective. 


Recommended  Research  Objectives 

The  results  of  the  interviews  were  clustered  in  like-type  categories  in  which  the  research 
objectives  were  derived.  Not  every  topic  fit  nicely  into  a  category  type,  those  topics  are 
captured  in  the  “other”  category.  The  research  objectives,  with  brief  descriptions,  are: 


•  Tradespace  -  Develop  and  communicate  analysis  of  requirements  tradeoffs  affecting 
system  design  for  decision  makers. 

•  Analytics  -  Expand  current  capability  to  leverage  meaningful  patterns  from  new  sources 
and  forms  of  data. 

.  CXXI,  AWARS,  LBC  -  Enhance  COMBATXXI,  AWARS  and  LBC  to  further  model 
additional  military  systems. 

•  OneTRAC  -  Investigate  opportunities  to  coordinate  TRAC  activities  with  other  agen¬ 
cies  and  across  Centers. 

•  Human  Dimensions  -  Account  for  the  effects  of  how  humans  function  in  a  system 
for  representation  in  modeling  and  analysis  as  well  as  increasing  performance  and 
innovation. 

•  Cyber  -  Categorize,  represent,  measure  and  analyze  cyber  effects  in  analysis  to  include 
simulation. 

•  Other  -  Examples  include  subterranean  effects,  utility  curves,  value  functions,  aviation 
acquire  algorithms  among  other  t6opics  that  did  not  fit  one  of  the  previous  categories. 


There  were  12  survey  responses  which  identified  multiple  research  requirements.  They  were 
counted  as  one  vote.  There  were  19  Interview  sessions  across  the  TRAC  Centers  (HQ  -  2, 
FLVN  -  7,  WSMR  -  7,  LEE  -3).  The  interviews  and  surveys  can  span  multiple  objectives, 
but  multiple  research  requirements  identified  within  the  objective  identified  by  the  surveys  or 
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a  single  interview  session  are  not  counted  separately  and  are  not  represented  multiple  times 
within  the  same  objective.  Figure  [l]  displays  the  interest  expressed  by  research  objective 
while  Figure  [2]  displays  interest  in  objective  by  TRAC  center. 


Figure  1.  Research  Objective  Interest  by  Vote 


Analytics 

Cyber 

CXXI, 

AWARS, 

LBC 

Trade 

space 

Human 

Dimensions 

OneTRAC  Other 

TRAC-HQ 

1* 

1* 

FLVN  4* 

3 

8 

4* 

2* 

5*  2 

LEE 

3* 

WSMR  7* 

3 

6 

7* 

2* 

4*  1 

Survey  3 

1 

4 

2 

3 

3  7 

*Center  director  emphasis 


Figure  2.  Research  Objective  Interest  by  TRAC-Center 


Supporting  Topics 

During  the  interviews  and  surveys,  there  were  86  topics  recommended  as  research  require¬ 
ments  of  which  61  were  unique  topics.  Binning  the  topics  by  research  objective  without 
regard  for  which  venue  suggested  the  topic  results  in  Figure  [3j 

The  specific  topics  supporting  the  category  type  are  listed  in  Appendix  |Aj 
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Figure  3.  Results  by  Topic 
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Chapter  4 

Conclusions  and  Lessons  Learned 


Approval  Process 

The  results  of  the  research  elicitation  were  presented  to  the  TRAC  Research  council  for 
consideration  during  the  research  council  meeting  hosted  by  TRAC-MTRY  on  the  19th  and 
20th  of  August,  2014.  The  research  council  discussed  the  results  and  agreed  to  present  the 
recommended  TRAC  Research  Objectives  in  Chapter  [3]  to  the  BOD  at  their  next  scheduled 
meeting. 

The  BOD  was  briefed  on  the  results  of  the  research  elicitation  along  with  the  recommended 
FY15  Research  Objectives  with  descriptions  during  their  meeting  hosted  by  TRAC-MTRY 
on  the  24th  of  September,  2014.  The  BOD  approved  the  FY15  TRAC  Research  Objectives 
to  be  incorporated  into  the  FY15  TRAC  Research  Plan. 


Recommendations  for  Future  Efforts 

Maintain 

With  the  dissolution  of  the  TRAC-MRO,  MTRY  is  the  lead  for  conducting  the  annual  re¬ 
search  elicitation  for  TRAC.  It  is  recommended  to  maintain  this  elicitation  annually  through 
surveys  and  in-person  interviews.  We  also  recommend  having  the  MTRY  Director  and  Re¬ 
search  Coordinator  present  for  all  interviews  during  the  process  because  it  helped  to  have 
two  sets  of  notes  to  compare  throughout  the  year. 

For  organizational  purposes,  further  recommendations  for  maintaining  are  listed  below: 


•  Using  a  distributed  survey  available  to  all  TRAC  employees 

•  Conducting  a  MTRY  Director’s  workforce  brief  at  each  center 

•  Interviewing  TRAC  leadership  in  flexible  settings 

•  Working  through  the  center  operations  to  schedule  the  interviews  with  each  direc¬ 
torate/division 

•  Executing  the  same  time  line  presented  in  Chapter  [l] 
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Changes  or  Additions 


Every  year  TRAC  conducts  the  research  elicitation  allows  for  opportunities  to  continually 
improve  the  process.  In  this  section,  we  recommend  changes  or  additions  that  will  enhance 
the  elicitation  effort  for  the  next  year.  A  major  addition  would  be  using  the  elicitation 
as  an  opportunity  to  show  the  progress  towards  research  objectives  since  last  year  to  the 
leadership  during  the  interviews  or  during  the  center  workforce  addresses.  Another  addition 
we  recommend  is  to  include  the  following  FY  TRAC-MTRY  work  plan  into  the  elicitation. 
By  the  spring  TRAC-MTRY’s  work  plan  for  the  next  fiscal  year  is  in  the  development 
stages.  The  research  interviews  are  an  opportunity  to  gage  the  interests  in  the  different 
project  proposals  for  the  next  year. 

Again,  for  organizational  purposes,  we  recommend  the  following  changes  or  implementations: 


•  Incorporating  the  re-established  offices  of  the  Principle  Analysts  at  FLVN  and  WSMR 

•  Gauging  TRAC-MTRY  work  plan  topical  interest 

•  Distributing  the  questionnaires  prior  to  the  interviews 

•  Conducting  a  mid-year  review  during  the  April  BOD 

•  Framing  the  elicitation  effort  within  the  TRAC  Strategy 

•  Providing  a  feedback  loop  to  the  centers  for  current  research 

•  Typing  a  Memorandum  For  Record  (MFR)  within  a  week  of  completing  the  interviews 
which  captures  who  was  at  the  session  and  topics  covered.  This  should  be  sent  to  the 
attendees  for  comment  to  ensure  completeness. 
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Appendix  A 

Research  Topics  by  Objective  Category 


These  short  titled  topics  were  captured  from  the  SharePoint  surveys  or  from  interviews 

Big  Data  Set  Exploration  Techniques 
Data  Fusion  of  hard  and  soft  information 
Data  VisualizationTechniques(RGIS) 

Design  Thinking  approach  to  problem  solving 
Text  Analytics 

Use  of  R  for  survey  analisys  and  text  mining 
Using  Open  Data  Sources 
Nano/Micro/Biotechnology  analysis 

Figure  A—l.  Analytics  Topics 
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LBC  Interface  for  AWARs 

Optimization  for  Criteria  to  tasking  queue  in  AWARs 
Aviation  asset  priorities  (medevac/casevac)  in  AWARs 
Battle  Damage  Assessment  methodology  AWARS 
Dynamic  convoy  routing  based  on  prior  attacks 
Post  processing  of  AWARS  charts  (Visualization) 

Short  Range  Observe  (SRO)  switching  with  regular  search  COMBATXXI 
Entity  behavior  in  Urban  OE  in  COMBATXXI 

military  effects  against  non-acquirable  urban  targets  in  COMBATXXI 
Management  of  multiple  course  of  action  via  SA-Triggers  COMBATXXI 
Misidentification  representation  in  COMBATXXI 
Mapping  target  fusion  within  simulation  in  COMBATXXI 
Acquisition  for  simulation  3rd  gen  (COMBATXXI) 

Using  COMBATXXI  as  Monte  Carlo  simulation  to  build  distributions 
Including  LBC  in  AWARS 

Risk  assessment  using  modeling  and  simulation 

LBC  Attrition  Capability 

LBC  Facility  Location  Tool 

LBC  Decision  Support  Tool  (Web  based) 

GPS  Jamming  in  COMBATXXI 
Combat  model  data  use 


Figure  A-2.  COMBATXXI,  AWARS,  LBC  Topics 


Cyber  Analysis 
Layered  Network  Effects 
Modeling  cyber  in  simulations 

Figure  A— 3.  Cyber  Topics 


Agent  Based  modeling 
Human  cognitive  behaviors  and  dimensions 
Improve  Decision  making  in  Models 
Leveraging  upcoming  trends  (innovation) 
Optimizing  Soldier/leader  performance 
Human  dimension  Analysis 


Figure  A— 4.  Human  Dimensions  Topics 
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Decision  Analysis  Support  Tools  (Value  Focus) 
Methodology  for  analyzing  across  DOTMLPF 
Trade  space  assessment  development 
F2025-holisticacquisition  and  force  design 
Operational  joint  cost  model 
Merging  Competing  interests  (Cluster) 

Figure  A— 5.  Tradespace  Analysis  Topics 


OneTRAC Searchable  Library 

Integration  of  studies  into  POM/CPR/LiRA  cycle 

KM  Development 

Measurement  Space  Drill  Support 

Front-end-analysis 

Integrating  Joint  considerations  into  studies 
JTF  Structure  and  SOF/Conventional  mix 

Figure  A  6.  OneTRAC  Topics 


Updated  sustainment  rules  for  TAA  (SOF) 

Agent  Based  modeling. 

Aviation  Acquire  Algorithm 
Computer  aided  mapping  exercise  improvement 
How  to  create  future  unit  MTOEs  using  USAFMSA 
Low-level  Campaign  Model 

Modeling  entity  behavior  in  response  to  network  (WINN-T 
SPADES  Technical  Review 
Streamline  Combat  Model  Runs 
Subterranean 

Utility  Curves  (Value  Functions) 


Figure  A— 7.  Other  Topics 
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Appendix  B 

SharePoint  Survey  Raw  Results 
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In  one  sentence,  briefly  summarize  a 
research  requirement  in  the  areas  of  topics, 
techniques,  and  methodologies. 

Please  describe  in  further  detail  the  need  identified. 

Which  TRAC 
center(s)/Directorate 
would  most  likely  be  a 

stakeholder? 

With  regards  to 
OneTRAC,  What 
is  the  priority  of 
this  research? 

Agent  Based  modeling. 

Exploration  of  agent  based  modeling  and  how  it  might  highlight  different 
results  than  existing  techniques.  Might  allow  us  to  answer  different  kinds 
of  decision  maker  questions. 

OneTRAC 

Moderate  -  High 

Force  2025  methodologies  -  The  Army 
approach  to  provide  an  umbrella  to  our  force 
design. 

The  Army  has  many  disparate  approaches  to  justifying  acquisition  needs, 
force  structure  and  funding  levels.  There  should  be  a  methodology  in 
which  this  is  done  in  a  concerted  effort. 

OneTRAC;  FLVN;  LEE; 

WSMR 

Moderate  -  High 

Sustainment  unit  workload  -  capability  factors 

The  TAA  process  uses  a  lot  of  "existance  rules"  for  various  sustainment  - 
logistics  unit  capabilites  to  support  the  force.  If  there  were  better  workload 
based  factors  available,  then  the  sustainment  -  logistics  force  structure  to 
support  a  given  operational  force,  especially  SOF,  could  be  better  tailored 
to  the  operational  force.  As  it  stands  now,  a  generic  SOF  operational  force 
will  be  supported  by  a  full  CSSB.  There  are  some  legacy  systems  like 
FASTALS  that  can  "generate"  a  sustainment  -  logistics  force,  but  it  can 
rapidly  spiral  out  of  control. 

LEE 

Moderate  -  High 

Cluster  Munitions 

1  heard  at  the  last  TRAC  workforce  address  that  this  might  be  an  upcoming 
project.  1  don't  think  TRAC  has  existing  research  done  to  frame  the  issue. 
Legal,  policy,  international  implications.  Might  be  a  research  requirement 
for  the  near  future.  Might  also  be  a  template  for  how  TRAC  might  work  in 
the  future  topics  (merging  different  and  sometime  competing 
perspectives)  and  assisting  with  decision  analysis  based  on  those 
competing  interests. 

WSMR; 

Moderate  -  Low 

Utility  Curves 

In  order  to  receive  warfighter  input  on  the  'good  enough'  capabilities  of 
gap  attributes,  irrespective  of  the  technology  solution,  a  recent  study  team 
applied  the  utility  curve  technique.  In  an  event  called  a  Warfighter  Utility 
Workshop,  the  team  developed  utility  curves  (also  called  value  curves  or 
value  functions).  To  my  knowledge,  there  is  nothing  in  terms  of  a  TRAC  SoP 
or  CoBP  or  diliberate  research  concerning  this  topic. 

It  would  be  helpful  for  study  leads  and  teams  to  understand  what  a  utility 
curve  is,  whether  it  is  appriate  for  their  study  or  not,  what  a  utility  curve 
workshop  should  look  like  and  what  kind  of  results  to  expect,  and  how  to 
use  the  results  to  inform  various  efforts  throughout  the  study. 
Additionally,  there  needs  to  be  sufficient  research  behind  techniques  that 
we  use  in  our  studies  in  order  to  keep  our  analysis  defensible. 

FLVN 

Moderate  -  High 

COMBATXXI  Urban  Operations  -  Movement, 
Search  and  Effects  in  very  complex  urban 
environment. 

A  new  COMBATXXI  tool  development  that  will  enhance  scenario  building 
clearing  capabilities  already  developed  using  Python  Scripts  to  an 
operational  database  behavior  methodology  per  the  current  Maneuver 
Tool  (MT)  methodology  that  will  be  oriented  towards  the  Urban 
Operational  Environment  (OE). 

WSMR 

Moderate  -  Low 

1.  Script-like  management  of  entity/unit  formation  movement  to  approach 
buildings.  Movement  through  buildings,  Movement  through  tunnels  and 
UGFs.  Need  to  be  able  to  maneuver  and/or  move  unit,  units,  individual  or 
individuals  (Team/Squad/Platoon/  Soldier/Buddy  Team/Manned 
Vehicle/Unmanned  Ground  Vehicle  (UGV)/Unmanned  Arial  System  (UAS)) 
though  out  the  Urban  OE  in  all  of  its  complexities.  These  maneuvers  should 
include  movement  through  buildings  (exterior  doors  and  windows, 
hallways,  interior  doors,  individual  rooms,  stairs,  hole  drops/climbs, 
dynamically  created  wall  breaches  and  rooftops),  through  UGFs/UGTs 
(entrances,  halls,  rooms,  hole  drops/climbs  and  stairs),  between  buildings 
(tunnels,  courtyards,  alleys,  rooftops  and  dynamic  breaches)  and  around 
urban  clutter  (vehicles,  exterior  walls,  junk  piles,  etc.).  Need  to  manage  the 
coordination  of  these  maneuvers  at  all  levels  (unit,  units,  individual  or 
individuals).  The  coordination  should  include  leadership  roles  (unit 
commanders),  unmanned  controllers  (entity  equipment)  and  ad-hoc 
networks  (dynamically  change  network  assignments).  In  addition  to 

COMBATXXI  Urban  Operations  -  Movement, 
Search  and  Effects  in  very  complex  urban 
environment. 

2.  Short  Range  Observe  (SRO)  switching  with  regular  search.  Need  to 
manage  which  search  technique  is  employed  depending  on  situation. 
Regular  search  when  outside  of  building.  SRO  search  when  conducting 
clearing  operations  indoor  or  interacting  with  enemy  indoors.  Regular 
search  when  inside  of  building  but  observing  targets  outside  of  building. 
And  default  over-rides  for  unique  situations. 

WSMR 

Moderate  -  Low 
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COMBATXXI  Urban  Operations  -  Movement, 
Search  and  Effects  in  very  complex  urban 
environment. 

3.  Applying  Effects.  Need  to  apply  military  effects  against  non-acquirable 
urban  targets  such  as  dynamic  breaches  (walls,  floors,  ceilings  and  closed 
doors),  pyrotechnics  (smoke,  flash  bangs  and  small  explosives)  and  self¬ 
illumination  (rooms,  halls  and  tunnels).  These  effects  could  be 
predetermined  (munitions-target  point  pairings)  and  executed  per  decision 
making  process. 

WSMR 

Moderate  -  Low 

COMBATXXI  Urban  Operations  -  Movement, 
Search  and  Effects  in  very  complex  urban 
environment. 

4.  Management  of  multiple  course  of  action  via  SA-Triggers.  Need  to 
determine  which  course  of  action  (COA)  to  select  at  unit  or  entity  level  per 
dynamic  decision  making.  Capability  should  be  part  of  UMT  rather  than 
executing  a  specific  BSL.  Methodology  may  include  COP  interrogation, 
CBUS,  flag  checking  or  other  defined  method  (e.g.,  HTN  recalculation 
event).  Processes  would  be  set  up  to  monitor  information  and  set  trigger 
events  within  UMT  that  would  execute  COA  at  decision  point  or  interrupt 
current  COA  (e.g.,  disengagement  criteria  triggering  withdrawal  or 
surrender).  Detailed  logging  of  information  and  decisions  will  be  required 
for  debugging  and  analysis. 

WSMR 

Moderate  -  Low 

SPADES  Lexicographical  Interpreter  Review 

System  for  Periodically  Apportioning  Demands  (SPADES)  is  a  "supply  and 
demand"  tool;  used  for  sufficiency  analysis  of  examining  Army  forces  ability 
to  meet  demand  signals  (i.e.  BCTs  versus  global  requirements).  The 
purpose  of  this  review  would  be  to  ensure  SPADES  is  functioning  properly 
(mini  V&V).  Task  would  be  to  review  the  SPADES  engine  and  report  any 
logic  areas. 

OneTRAC 

Moderate  -  High 

Test  -  Data  Mining 

Application  of  data  mining  to  support  TRAC  analysis.  What  is  the  breadth 
of  open  source  data  they  we  may  be  able  to  access  /  leverage  to  support 
TRAC  analysis?  What  tools  does  TRAC  have  to  incorporate  these  sources? 
What  are  the  risks  of  using  this  type  of  data  source? 

OneTRAC 

Moderate  -  High 

Streamlining  the  setup  of  combat  models 

Study  teams  wishing  to  use  our  major  combat  models  (CXXI,  etc)  currently 
need  a  lead  time  of  months  for  the  runs  to  be  completed.  Modernized 
techniques  might  be  able  to  further  streamline  the  process. 

OneTRAC;#FLVN;#WSM 

R 

Moderate  -  Low 

Text  Analytics  Capability 

The  organization  needs  appropriate  tools  and  knowledge  sharing  to 
support  rapid  text  analysis.  At  TRAC-WSMR  we  have  a  recurring  need  for 
text  analytic  tools  to  evaluate  large  amounts  of  text.  Project  time 
constraints  frequently  prevent  analysts  from  developing  the  text  analytic 
tools  and  techniques  in  stride  with  project  demands.  For  a  time,  TRAC- 
WSMR  purchased  an  add-on  text  analyzer  module  for  PASW  Modeler,  but 
the  learning  curve  was  steep,  and  there  was  insufficient  training  available. 
The  software  is  no  longer  licensed  at  TRAC-WSMR.  In  the  past,  1  or  my 
subordinates  have  needed  such  tools  for  evaluating  open-ended  survey 
responses,  FBCB2  chat  and  other  military  communications,  and,  most 
recently,  Twitter  feeds.  We  wanted  to  analyze  social  media  feeds  to 
identify  humanitarian  assistance  needs  in  Syria,  and,  assuming  the  findings 
bore  fruit,  eventually  develop  an  automated  utility  for  such  analysis  that 
could  be  used  by  non-ORSA  in  CENTCOM-Jordan. 

OneTRAC 

Moderate  -  High 

One  TRAC  does  not  currently  have  the  ability 
to  search  documents,  reports,  research,  or 
other  products  across  its  centers. 

Research  options  for  a  searchable  library/database  to  house  TRAC 
products  on  both  classified  and  unclassified  networks.  The 
library/database  should  include  have  the  ability  to  search  key  words  and 
topics.  Is  should  also  capture  integrate  historical  documents  that  may  only 
be  in  paper  format. 

OneTRAC 

Moderate  -  High 

Apply  analytical  techniques  to  minimize 
threat  to  rotary  wing  aircraft  in  a  multi¬ 
faceted  threat  environment. 

There  does  not  seem  to  exist  any  analytical  approach  to  minimizing  the 
aggregate  threat  to  rotary  wing  aircraft  from  small  arms,  AAA,  IR  and  Radar 
SAMs  (i.e.  at  what  altitude  is  the  total  threat  value  minimized?) 

FLVN 

Moderate  -  Low 

Big  data. 

Continue  to  explore  techniques  to  explore  big  data  sets  that  might  allow  us 
to  address  different  questions  from  decision  makers. 

OneTRAC 

Moderate-High 

Innovation 

In  MRO,  we  are  looking  at  this  topic  from  both  a  micro  level-How  can  TRAC 
use  innovation  to  continue  to  be  a  strong  organization  and  a  MACRO  level-- 
what  is  the  next  "big  thing"  wrt  innovation.  There  is  a  body  of  social 
science  that  looks  at  the  topic,  but  how  do  we  leverage  this  to  help  our 
analysts  do  a  better  job  on  a  day  to  day  basis,  while  spoting  key  trends 
(next  big  thing)  for  future  analytical  work? 

OneTRAC 

High 
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Identify  minimum  requirements  for  Theater 
level  combat  simulation 

Currently  AWARS  can  simulate  up  to  Div/Corps/JTF.  Several  recent  efforts 
have  wanted/needed  a  theater  or  campaign  level  combat  outcomes. 
Typically  this  would  be  a  CAA  area  but  they  have  challenges  with  turn 
around  time  and  capacity  due  to  TAA  requirements.  We  have  been 
discussing  ideas  on  how  to  get  at  a  low  resolution  campaign  tool  that 
provides  acceptable  (i.e.  not  replacing  JICM/COSAGE  but  something  in 
between  JICM  and  AWARS)  results. 

FLVN 

Moderate-Low 

Improving  Cognitive  Decision  Making  in  Army 
Models  and  Simulations 

Examine  decision-making  and  leadership  research  to  determine  possible 
gaps  between  academic  theories  and  modeling  and  simulation  efforts. 

Seek  to  improve  decision  processes  in  current  TRAC  models. 

OneTRAC 

Moderate-High 

Cyber  topics. 

Cyber  can  affect  our  soldiers/plans  beyond  just  how  they  affect  us  on  the 
battlefield.  Robust  cyber  analysis  can  allow  us  to  field  cyber  related 
questions/analysis. 

OneTRAC;  FLVN 

Moderate-High 

"Front  End  Analysis" 

Recent  Gender  Study  required  a  high  level  of  "front  end  analysis"  1  can  see 
where  we  did  a  good  job  with  some  of  our  front  end  analysis,  and  am 
aware  of  places  we  could  have  done  a  better  job.  How  are  other  study 
leads  using  front  end  analysis?  How  can  this  process  be  expedited?  Should 
it  be  expedited?  Are  we  doing  it  right? 

OneTRAC 

Moderate-High 

Subterranean  Operations. 

The  topic  seems  to  be  one  spanning  from  SOCOM  over  to  the  conventional 
force.  We  might  want  to  start  developing  scenarios  and  M&S  capability 
that  will  allow  for  this  type  of  analysis. 

OneTRAC 

Moderate-Low 

Model  JTF 

Why  do  certain  JTF  structures  work  well  and  others  fail  to  make  progress  in 
their  AOR?  How  can  a  JTF  be  structured  to  work  well  with  joint, 
interagency,  international  and  host  nation  authorities?  What  is  the 
optimal  interplay  between  conventional  forces  and  special  operations 
forces  to  increase  chances  for  successful  mission  accomplishment?  This  is 
something  that  could  be  done  in  conjunction  with  CAA. 

OneTRAC 

Moderate-High 
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Appendix  C 
Interview  Notes 


To  guide  the  interviews  we  used  a  questions  to  frame  the  interviews  but  let  the  flow  be 
driven  b  topic  rather  than  a  prescriptive  formula  for  each  interview.  The  questionnaire  is 
displayed  in  Appendix  [D]  In  this  appendix,  we  display  the  topics  as  they  were  discussed  in 
the  interviews  of  each  directorate/division  of  each  center. 


TRAC-HQ  Interviews  Notes 

Table  C— 1.  Director,  TRAC 


Type  Topic 

Human  Dimensions  Human  dimension  Analysis 
OneTRAC  Measurement  Space  Drill  Support 


TRAC-FLVN  leader  Interview  Notes 

Table  C— 2.  FLVN-Director 


Type  Topic 

Analytics  Data  Visualization  Techniques  (RGIS) 

Human  Dimensions  Optimizing  Soldier/leader  performance 
Tradespace  F2025-holistic  acquisition  and  force  design 

OneTRAC  Measurement  Space  Drill  Support 


Table  C— 3.  FLVN-Deputy  Director 


Type  Topic 

Analytics  Data  Fusion  of  hard  and  soft  information 

Tradespace  Decision  Analysis  Support  Tools  (Value  Focus) 

Human  Dimensions  Human  cognitive  behaviors  and  dimensions 
Analytics  Nano/Micro/Bio  technology  analysis 

Cyber  Cyber  Analysis 
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Table  C— 4.  FLVN-Analysis  Directorate 


Type 

CXXI,  AWARS,  LBC 

Analytics 

OneTRAC 

Tradespace 

Cyber 


Topic 

Combat  model  data  use 

Big  Data  Set  Exploration  Techniques 

OneTRAC  Searchable  Library 

Decision  Analysis  Support  Tools  (Value  Focus) 

Cyber  Analysis 


Table  C— 5.  FLVN-Studies  Directorate 


Type  Topic 

Tradespace  Decision  Analysis  Support  Tools  (Value  Focus) 


Table  C— 6.  FLVN-Wargaming  and  Simulation  Directorate 


Type 

CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
Other 

CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 


Topic 

LBC  Interface  for  AWARs 

Optimization  for  Criteria  to  tasking  queue  in  AWARs 
Aviation  asset  prioirities  (medivac/casevac)  in  AWARs 
Computer  aided  mapping  exercise  improvement 
Battle  Damage  Assessment  methodology  AWARS 
Dynamic  convoy  routing  based  on  prioir  attacks 
Postprocessing  of  awars  charts  (Visualization) 
Including  LBC  in  AWARS 


Table  C— 7.  FLVN-Operations  Directorate 


Type 

OneTRAC 

OneTRAC 

OneTRAC 


Topic 

KM  Development 
Measurement  Space  Drill  Support 
OneTRAC  Searchable  Library 


Table  C— 8.  FLVN-Scenarios  and  Data  Directorate 


Type  Topic 

Other  How  to  create  future  unit  MTOEs  using  USAFMSA 
Cyber  Modeling  cyber  in  simulations 
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TRAC-WSMR  Leader  Interview  Notes 


Table  C— 9.  WSMR-Director 


Type 

Topic 

OneTRAC 

Integration  of  studies  into  POM/CPR/LiRA  cycle 

OneTRAC 

Measurement  Space  Drill  Support 

Analytics 

Text  Analytics 

Human  Dimensions 

Human  cognitive  behaviors  and  dimensions 

Tradespace 

Methodology  for  analyzing  across  DOTMLPF 

OneTRAC 

OneTRAC  Searchable  Library 

Table  C-TO.  WSMR-Deputy  Director 

Type 

Topic 

Tradespace 

Methodology  for  analyzing  across  DOTMLPF 

Tradespace 

F2025-holistic  acquisition  and  force  design 

Table  C-ll. 

WSMR-Studies  and  Analysis  I  Directorate 

Type 

Topic 

CXXI,  AWARS,  LBC  GPS  Jamming  in  COMBATXXI 

Tradespace 

Tradespace  assessment  develpoment 

OneTRAC 

Integrating  Joint  considerations  into  studies 

Table  C-12. 

WSMR-  Studies  and  Analysis  II  Directorate 

Type 

Topic 

Tradespace 

Methodology  for  analyzing  across  DOTMLPF 

Analytics 

Use  of  R  for  survey  analisys  and  text  mining 

Human  Dimensions  Optimizing  Soldier/leader  performance 

Tradespace 

Tradespace  assessment  develpoment 

Analytics 

Text  Analytics 
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Table  C—13.  WSMR  -  Studies  and  Analysis  III  Directorate 


Type 

Analytics 

Other 

Cyber 

CXXI,  AWARS,  LBC 

Analytics 

Analytics 


Topic 

Design  Thinking  approach  to  problem  solving 

Modeling  entity  behavior  in  response  to  network  (WINN-T) 

Layered  Network  Effects 

Risk  assessment  using  modeling  and  simulation 
Data  Fusion  of  hard  and  soft  information 
Text  Analytics 


Table  G-T4.  WSMR  -  Modeling  and  Simulation  Directorate 


Type 

CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 
CXXI,  AWARS,  LBC 


Topic 

Missindentihcation  respresentation  in  CXXI 
Mapping  target  fusion  within  simulation  in  CXXI 
Acquisition  for  simulation  3rd  gen  (CXXI) 

Using  CXXI  as  monte  carlo  simulation  to  build  distributions 


Table  C— 15.  WSMR-  Study  Support  Directorate 


Type 

Topic 

Tradespace 

Operational  joint  cost  model 

Cyber 

Layered  Network  Effects 

Cyber 

Modeling  cyber  in  simulations 

Analytics 

Text  Analytics 

C-4 


TRAC-LEE  Leader  Interview  Notes 


Table  C— 16.  LEE-Director 


Type  Topic 

CXXI,  AWARS,  LBC  LBC  Decision  Support  Tool  (Web  based) 


Table  C— 17.  LEE-Modeling  and  Analysis  Division 


Type  Topic 

CXXI,  AWARS,  LBC  LBC  Attrition  Capability 
CXXI,  AWARS,  LBC  LBC  Facility  Location  Tool 
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Appendix  D 
Questionnaire 


FY15  TRAC  Research  Requirements  Elicitation  Interviews 


Describe  a  research  requirement  in  the  areas  of  topics,  techniques,  and  method¬ 
ologies. 


Which  TRAC  center(s)/Directorate  would  most  likely  be  a  stakeholder? 


With  regards  to  OneTRAC,  What  is  the  priority  of  this  research? 


D-l 


What  are  your  top  three  priorities  for  this  next  FY? 


Of  the  research  requirements,  what  priority  order  would  you  assign  this  need? 


How  relevant  are  the  current  research  objectives?  What  about  in  the  context  of 
2025? 


What  dates  in  the  past  year  did  you  have  a  measurement  space  drill  and  what 
venue  was  used  (onsite,  VTC,  DCO,  etc.)? 


Please  make  sure  to  complete  survey  on-line. 
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Appendix  F 
Glossary 

AWARS 

Advanced  Warfare  Simulation 

BCT 

Brigade  Combat  Team 

BOD 

Board  of  Directors 

EEA 

Essential  Elements  of  Analysis 

FLVN 

Fort  Leavenworth 

FY 

Fiscal  Year 

KM 

Knowledge  Management 

LBC 

Logistics  Battle  Command 

LEE 

Fort  Lee 

M&S 

Modeling  and  Simulation 

MADM 

Multi- Attribute  Decision  Making 

MRO 

Methods  and  Research  Office 

MTRY 

Monterey 

SES 

Senior  Executive  Schedule 

SMA 

Senior  Military  Analyst 

SME 

Subject  Matter  Expert 

TRAC 

Training  and  Doctrine  Command  Analysis  Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 
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